IN THE CLAIMS 



1. (Currently amended) A method for providing collateral information for 
inclusion with an information stream, comprising steps of: 

examining the information stream to recognize a presence of events that 
occur in the information stream, wherein said events are derived from the 
information stream based on one or more predetermined taxonomies , wherein 
the step of examining the information stream comprises the steps of 
automatically extracting text from the information stream, segmenting the text 
into sentences and a step of operating on the sentences to identify topics that 
correspond to topic taxonomies of the predetermined taxonomies and the 
presence of names of entities; 

assembling a list comprised of an identified topic having a start time and 
an end time, as well as any named entities that occur between the start time and 
the end time; 

assembling a guerv object comprised of named entities that occur 
between the start time and the end time of the identified topic; 

searching at least one database to identify a first set of stored documents 
that correspond to the topic; 

identifying a subset of the first set of documents that contain the named 
entities; 

identifying a second set of documents that correspond to words found in 
the text; 
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scoring the returned documents based on a plurality of criteria and ranking 
the documents based on their scores ; 

automatically generating database queries from said derived events; and 

analyzing results of said database queries so as to rank and select said 
results to be inserted into the information stream as information that is collateral 
to said derived events. 

2. Canceled 

3. (Currently amended) A method as in claim 21, wherein the plurality of 
criteria comprise a score derived from a free text search of the database using 
text that is automatically extracted from the information stream, on a number of 
named entities appearing in the text and in the database query results, and on a 
taxonomy path score, where the taxonomy path score represents an amount of 
relatedness between a taxonomy-related information element found in the text 
and a tree of the predetermined taxonomies. 

4-6. (Canceled) 

7. (Currently amended) A method as in claim §1 , wherein the step of 
automatically extracting text from the information stream comprises a step of 
operating a voice recognition system. 

8. (Currently amended) A method as in claim §1, wherein the step of 
automatically extracting text from the information stream comprises a step of 
extracting closed caption text. 
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9. (Currently amended) A method as in claim §1, wherein the step of 
automatically extracting text from the information stream comprises a step of 
operating a character recognition system. 

10. (Currently amended) A method as in claim §1, wherein the step of 
automatically extracting text from the information stream comprises a step of also 
generating text that is descriptive of a number of human faces that are present in 
an image conveyed by the information stream. 

1 1 . (Canceled) 

12. (Currently amended) A method for providing collateral information for 
multiplexing with an information stream, comprising steps of: 

converting the information stream into text; 

analyzing the text to identify information elements based on one or more 
predetermined taxonomies; 

automatically generating queries from the information elements for 
searching at least one database; 

extracting data from database search results that is relevant to the 
information stream , wherein the step of extracting comprises a step of ranking 
extracted document information based on a score derived from a free text search 
of a document database using the text, on a number of named entities extracted 
from the text that are found in the documents, and on a taxonomy path score, 
where the taxonomy path score represents an amount of relatedness between a 
taxonomy-related information element identified in the text and a tree of the 
predetermined taxonomies ; and 
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multiplexing the data into the information stream for presentation at a 
destination of the information stream. 

13. (Canceled) 

14. (Canceled) 

15. (Previously presented) A method as in claim 12, wherein the queries 
are generated based on information elements that correspond to a list of 
information elements identifying topics in the text being analyzed, where the 
topics correspond to topic taxonomies of the predetermined taxonomies. 

16. (Previously presented) A method as in claim 12, wherein the step of 
analyzing the text comprises steps of segmenting the text into sentences and a 
step of operating on the sentences to identify topics that correspond to topic 
taxonomies of the predetermined taxonomies, and wherein the step of 
automatically generating queries operates on identified topics. 

17. (Canceled) 

18. (Canceled) 

19. (Currently amended) A system for providing collateral information for 
inclusion with an information stream, said system operating in real time or 
substantially real time and comprising: 

a subsystem for examining the information stream to recognize a 
presence of events that occur in the information stream, wherein said events are 
derived from the information stream based on one or more predetermined 
taxonomies; 
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a subsystem, having an input coupled to an output of said examination 
subsystem, for automatically generating database queries from said derived 
events; 

a database for receiving said database queries; and 

a subsystem, having an input coupled to an output of said database, for 
analyzing results of said database gueries so as to rank and select said results to 
be inserted into the information stream as information that is collateral to said 
derived events as i n c l a i m 18 , wherein the analyzing subsystem employs ranking 
criteria comprised of a score derived from a free text search of the database 
using text that is automatically extracted from the information stream, on a 
number of named entities appearing in the text and in the database query results, 
and on a taxonomy path score, where the taxonomy path score represents an 
amount of relatedness between a taxonomy-related information element found in 
the text and a tree of the predetermined taxonomies, and wherein the query 
generation subsystem generates queries based on information corresponding to 
a list that identifies topics in the text that is automatically extracted from the 
information stream, where the topics correspond to elements of the taxonomy 
tree. 

20. (Previously presented) A system as in claim 19, wherein said 
examining subsystem comprises at least one unit for automatically extracting text 
from the information stream, a unit for segmenting the text into sentences and at 
least one unit for operating on the sentences to identify topics that correspond to 
topic taxonomies of the predetermined taxonomies, wherein said query 
generation subsystem automatically generates database queries based at least 
in part on identified topics. 
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21. (Original) A system as in claim 20, wherein said text extracting unit 
comprises at least one of a voice recognition system, a system for extracting 
closed caption text, and a character recognition system. 

22. (Original) A system as in claim 20, wherein said examining subsystem 
comprises a unit for generating text that is descriptive of a number of human 
faces that are present in an image conveyed by the information stream. 

23. (Currently amended) A system for providing collateral information for 
inclusion with an information stream, said system operating in real time or 
substantially real time and comprising: 

a subsystem for examining the information stream to recognize a 
presence of events that occur in the information stream, wherein said events are 
derived from the information stream based on one or more predetermined 
taxonomies, wherein said examining subsystem comprises at least one unit for 
automatically extracting text from the information stream, a unit for segmenting 
the text into sentences and at least one unit for operating on the sentences to 
identify topics that correspond to topic taxonomies of the predetermined 
taxonomies; 

a subsystem, having an input coupled to an output of said examination 
subsystem, for automatically generating database gueries from said derived 
events, wherein said guery generation subsystem automatically generates 
database gueries based at least in part on identified topics; 

a database for receiving said database gueries; 

a subsystem, having an input coupled to an output of said database, for 
analyzing results of said database gueries so as to rank and select said results to 
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be inserted into the information stream as information that is collateral to said 
derived events; and as in claim 20. and furth e r comprining 

a unit for operating on the sentences to identify the presence of names of 
entities, and further comprising a unit for assembling a list comprised of an 
identified topic having a start time and an end time, as well as any named entities 
that occur between the start time and the end time, and where the query 
generation subsystem assembles a query object comprised of named entities 
that occur between the start time and the end time of the identified topic for 
searching said database to identify a first set of stored documents that 
correspond to the topic, a subset of the first set of documents that contain the 
named entities, a second set of documents that correspond to words found in 
the text; and where said analyzing subsystem scores the returned documents 
based on a plurality of criteria and ranks the documents based on their scores. 

24-32. (Canceled) 

33. (Currently amended) A computer readable media having recorded 
thereon a program for providing collateral information for inclusion with an 
information stream, the program comprising instructions for: 

examining the information stream to recognize a presence of events that 
occur in the information stream, wherein the events are derived from the 
information stream based on one or more predetermined taxonomies , wherein 
the instruction for examining the information stream comprises instructions for 
automatically extracting text from the information stream, for segmenting the text 
into sentences and for operating on the sentences to identify topics that 
correspond to topic taxonomies of the predetermined taxonomies and the 
presence of names of entities: 
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assembling a list comprised of an identified topic having a start time and 
an end time, as well as any named entities that occur between the start time and 
the end time; 

assembling a guery object comprised of named entities that occur 
between the start time and the end time of the identified topic; 

searching at least one database to identify a first set of stored documents 
that correspond to the topic; 

identifying a subset of the first set of documents that contain the named 
entities; 

identifying a second set of documents that correspond to words found in 
the text; 

scoring the returned documents based on a plurality of criteria and ranking 
the documents based on their scores; 7-fef 

automatically generating database queries from said derived events; and 

fef 

analyzing results of said database queryies r e su l ts so as to rank and 
select database guerv said results to be inserted into the information stream as 
co ll at e ra l information that is collateral to said derived events. 
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